This specification relates to parallel computing.
In parallel computing, a large program can be divided into multiple smaller ones. The smaller programs are distributed to multiple interconnected computers, and executed in parallel to one another. Parallel computing has been implemented using various frameworks, each including framework-unique protocols, libraries, tool sets, and application programming interfaces (APIs).
Some parallel computing frameworks are based on a message passing interface (MPI) standard. MPI standards generally include a set of protocols defining syntax and semantics of various library routines for message passing and specifying how programs behave in a parallel computing system. A program written according to an MPI standard can execute on a cluster of computers of a corresponding MPI framework.